The present invention relates to an image processing device and an image processing method.
In recent years, it is highly necessary that personal computers, gaming devices, digital-broadcast-ready television sets, Blu-ray recorders, and other digital home appliances have a three-dimensional (3D) capability. More specifically, it is necessary that these devices accept an input 3D image and properly display the 3D image.
The 3D image input format for the above-mentioned devices has been diversified. The 3D image output format for the devices has also been diversified. In consideration of the diversified 3D image input/output format, some models of television devices having a 3D image display capability have newly incorporated various 3D signal processing functions in addition to a 3D image display function compliant with an existing digital broadcast standard. However, these models have entailed increased cost, including the cost of development and manufacture and the cost of overall system, because they have special functions such as the 3D signal processing functions.
A method of generating a 3D stereoscopic image, which is to be input into a display functionality of the above devices, is disclosed in US Patent Publication No. 2010/0302235. The method disclosed in US Patent Publication No. 2010/0302235 uses a pixel blender (synthesis means), such as a GPU (Graphics Processing Unit), to generate a 3D stereoscopic image from plural images (two images in most cases) and mask information.
FIG. 15 is a diagram illustrating a method of generating a 3D stereoscopic image in a 3D television system described in US Patent Publication No. 2010/0302235. A method of generating a 3D stereoscopic image 53 from a left eye image 51 and a right eye image 52 is described below. The 3D stereoscopic image 53 is structured so that the pixel values of pixels included in the left eye image are set for pixels in odd-numbered lines while the pixel values of pixels included in the right eye image are set for pixels in even-numbered lines.
The method described in US Patent Publication No. 2010/0302235 uses pixel blenders 60, 61, which are operated by a CPU or GPU. The pixel blender 61 reads the left eye image 51 and mask information 54 from a storage device such as a RAM (Random Access Memory). The pixel blender 60 performs a process defined by Porter-Duff's “A atop B” rule. More specifically, the pixel blender 60 sets the pixel value of a pixel in the corresponding position of the left eye image 51 for a pixel for which the value “1” is designated in the mask information 54. The pixel blender 60 does not set a pixel value for a pixel for which the value “0” is designated in the mask information 54. The pixel blender 60 generates an intermediate image 55 by performing the above-described process for all pixels. The pixel blender 60 writes the generated intermediate image 55 in the storage device such as a RAM.
The pixel blender 61 reads the right eye image 52 and the intermediate image 55 from the storage device such as a RAM. The pixel blender 61 performs a process defined by Porter-Duff's “A over B” rule. More specifically, the pixel blender 61 sets the pixel value of a pixel in the corresponding position of the right eye image 52 for a pixel for which the value “0” is designated in the intermediate image 55. The pixel blender 61 does not perform the process for a pixel for which the pixel value derived from the left eye image is already set. The pixel blender 61 generates an output image 53 by performing the above-described process for all pixels. The pixel blender 61 writes the generated output image 53 in the storage device such as a RAM.
FIG. 16 is a diagram illustrating the relationship between each process and memory access (access to the storage device) that occurs when the method described in US Patent Publication No. 2010/0302235 is used. The pixel blender 60 performs a process of reading the storage device three times. The pixel blender 61 performs a process of reading the storage device two times. The pixel blenders 60, 61 perform a process of writing into the storage device once. Further, another processing unit performs a process of reading the output image 53 from the storage device once.
Plane synthesis, which is described in “Data Broadcast Encoding Method and Transmission Method in Digital Broadcasting” (retrieved from the Internet on Apr. 10, 2011; URL: http//www.arib.or.jp/english/html/overview/doc/2-STD-B24v5—1-1p3.pdf), is suggested in Japanese Unexamined Patent Publications No. 2003-324652 and 2010-068399. A plane synthesis technology described in “Data Broadcast Encoding Method and Transmission Method in Digital Broadcasting” will now be described with reference to FIG. 17. The plane synthesis technology defines the plane synthesis for digital broadcasting.
FIG. 17 is a schematic diagram illustrating the digital broadcast plane synthesis technology described in “Data Broadcast Encoding Method and Transmission Method in Digital Broadcasting”. The term “plane” refers to a display screen for displaying mono-media (independent expression media such as video, audio, text, and still images).
A mixer unit 70 reads still image data 81 from a RAM 80 as video/still plane 0 (91). The mixer unit 70 reads moving image data 82 from the RAM 80 as video/still plane 1 (92). Video/still plane 0 (91) and video/still plane 1 (92) may be either a moving image or a still image.
Similarly, the mixer unit 70 reads text data 84 from the RAM 80 as a subtitle plane 94. The mixer unit 70 reads text/graphic data 85 from the RAM 80 as a text/graphic plane 95. Digital broadcast requirements prescribe that five planes be handled as shown in FIG. 17. However, a lager number of planes may be handled. For example, a plane for displaying a user's operating menu and a plane for displaying a cursor may be additionally handled.
The mixer unit 70 reads a switching plane 83 from the RAM 80. As regards the switching plane 83, the setup for a transparentizing process is defined for a situation where two planes (video/still plane 0 (91) and video/still plane 1 (92) in the current example) are synthesized. As for the switching plane 83, 1-bit data is set for each pixel. The 1-bit data is either a value (“0”) indicating the setup of a pixel value for video/still plane 0 (91) or a value (“1”) indicating the setup of a pixel value for video/still plane 1 (92).
The mixer unit 70 can independently set a scaling ratio for video/still plane 0 (91) and video/still plane 1 (92). In other words, the mixer unit 70 can individually enlarge or reduce video/still plane 0 (91) and video/still plane 1 (92) while keeping them independent of each other.
The mixer unit 70 reduces video/still plane 9 (91) in accordance with a region defined by the switching plane 83 (video/still plane 9 (91) in FIG. 17 is reduced to ½). Further, the mixer unit 70 examines a bit value that is set for each pixel of the switching plane 83. If the value “0” is set as the bit value for a particular pixel, the mixer unit 70 operates so that the pixel value of a target pixel of video/still plane 0 (91) is set for the particular pixel. If, on the other hand, the value “1” is set as the bit value for a particular pixel, the mixer unit 70 operates so that the pixel value of a target pixel of video/still plane 1 (92) is set for the particular pixel. In this manner, the mixer unit 70 generates an intermediate image (not shown) that is a combination of video/still plane 0 (91) and video/still plane 1 (92).
The mixer unit 70 superimposes the subtitle plane 94 and the text/graphic plane 95 over the intermediate image to generate an output image 96.
Although “Data Broadcast Encoding Method and Transmission Method in Digital Broadcasting” suggests and teaches the digital broadcast plane synthesis technology, it does not describe the generation of a stereoscopic image at all.